It has recently been made clear once more just how hard it is to get that first connection to the gnutella network because of its dynamic nature. Nodes with which you can connect are always coming and going and a good list of addresses today is of no value after a short time. To try to handle this issue a “turbo start” feature has been added to auto-connect. Whenever auto-connect is turned on, Mactella first goes to "gnutellahosts.com" where it can usually get a list of about ten good candidate sites. These sites are immediately tried. In general whenever Mactella gets a response from a live site somewhere on the network (a “pong packet”) that site moves to the front of the list of sites to try when we are still looking for direct connections.

The other main change is how names of files are shortened when the name used on the gnutella host is too long for the Mac file system. The extra characters are taken from the middle of the name so that if the name includes a useful extension (such as .mp3) that is preserved and it is more likely that the file while be mapped to the appropriate application.

A significant amount of work has been started on a Carbon version of Mactella. This application seems to run well in System 9 using CarbonLib but is still too unstable in OS X. If anyone would like to try the carbonized version just drop me a line by email (sbryan@cxc.com) but understand its limited usefulness currently in OS X.

It's beginning to look more like a month is the likely interval between releases. The current release is specifically a bug fix release although there are other changes as well. For quite a while there has been a crashing bug that prevented Mactella from running for extended periods of time. This appears to be associated with the strategy of reusing resources such as memory buffers. By re-organizing how this reuse is accomplished, it appears that a major source of otherwise mysterious crashes has been eliminated.

The other important change with this version of Mactella is that the new compiler from Metrowerks is now being used to build. The implication of this change is that a Carbon version of Mactella is now on the horizon.

Well, it has been more than a week, more than a month, but less than a season since the last release. There remains an elusive, annoying bug associated with uploading but a new release is long overdue.

The preferences dialog has been significantly expanded to allow the user more flexibility. The most important change allows, in some circumstances, a node behind a firewall to appear as though it has full two way connectivity to the rest of the net. The circumstance alluded to in the previous sentence is that the firewall software must map the 'gnutella' ports to the specific private address of the Mac that is running Mactella. The preferences also allow the 'gnutella' ports to be edited but by default they are 6344, 6345, 6346. So if your preferences are set correctly (ie you put in the correct IP address of the firewall machine and turn on the checkboxes) and the firewall is mapping the right ports, your client will appear to be running at the address of your firewall machine. So for one person per firewall we can finesse the NAT2NAT deficiencies of the gnutella protocol.

There are now two connection speed parameters: actual and minimum. Actual refers to the speed of your internet connection. The minimum speed is used to filter search results. If the connection speed of a server is less than the minimum speed that has been selected, its search results will be filtered out.

We will also filter out search results of servers that have an unroutable address (e.g. 192.168.x.x) if the user's machine is behind a firewall and it is not using an external IP address. Of course if it is using an external IP address (of the firewall and the gnutella ports are being routed) then we no longer need to filter out search results from unroutable addresses since there is still the possibility to use the push protocol to get at those files.

To control some screen clutter the preferences dialog uses a disclosure triangle to hide some of the fields that should usually be left alone. If you do choose to edit the port numbers that are used make certain you specify three distinct port numbers and restart the program. In future versions it should be possible to prevent colliding port numbers and allow changing the ports on the fly.

Another important change has to do with unroutable addresses and push protocol. In the past push packets were only sent to unroutable addresses. Now, even if the address appears to be reachable we will send out a push packet if the attempt to connect fails. This should help in cases where a firewall prevents an otherwise fine address from responding.

One of the most common questions about Mactella has to do with getting the downloaded file to load into the right application. In the past we tried in cases where a file name extension gave a hint, to set the file type and creator to appropriate values. But often there was no hint. Now if a downloaded file is still assigned to Mactella, just double click it. A window will open with the file name in its title. If you know the four letter type and creator codes just type them in and click the button to change the parameters. Even if you don't but you have an example file close by, just drag and drop it on the window. The codes will be set the same as that file and you can now click to set them for your downloaded file.

Many of the errors and inefficiencies have been wrung out of Mactella. But there is still a mysterious crashing bug that only seems to occur if uploading is enabled (by allowing more than zero uploads in the preferences menu). I think it is fair to speculate that this is due to a failure of Mactella to implement the file transfer compatibly with one of the other gnutella clones. Much of the current work will be to test and verify file transfer compatibility with the other clones in the hopes of extinguishing this remaining major bug. When that is accomplished we will be releasing version 1.0 which should be able to run 24/7.

The hardest thing to do is to stop coding and let the thing go out the door. Releasing every week is just too painful because there are always a few more modifications which will make things ever so much better. But enough is enough, here is the next version, the first beta release.

Mactella now supports push protocol file transfers and pays attention to how you set the checkbox for “Are you behind a firewall” in the prefs dialog. If you are not behind a firewall you will see search results from nodes that are on private networks (indicated by an address of the form 192.168.x.x). Instead of trying fruitlessly for a direct connection a push packet is sent requesting the node to connect to you and send the specific file. If you are on a private network and someone wants to get a file from you he should send a push request packet which will be handled by Mactella. This feature has been tested to work between Mactella hosts and has also been specifically verified for Furi, GNucleus, and Gnotella. Testing will continue with other clients and further elaborations will probably be required.

Mactella also now supports restarting interrupted transfers between Mactella hosts. It should work with all well behaved clients but I haven't found support for partial downloads with the few clients I've looked at. The way this currently works is that you double click a search result to start a file transfer. If the transfer stops before finishing you can attempt to restart it by hilighting the icon in the download panel and clicking the restart button. In the future it will be possible to attempt a restart by double clicking the partial file icon in the Finder but that aspect of the feature is not done yet.

Multiple file type support has been improved vastly by allowing for more than just one file type. Currently this is done specifically by examining the file name extension. In future versions of Mactella this capability will be extended considerably.

Filtering of search results by IP address is now initially implemented. Some clients have addresses set to unusable addresses like 0.0.0.0, 127.0.0.1, etc. Since downloads in such cases would fail they are filtered out. Also if you indicate that you are behind a firewall (ie your address is not directly reachable from those outside your local network) in the preferences dialog, then search results that are at unroutable addresses (of the form 192.168.x.x) are dropped since even a push request would fail to produce a connection. This feature will definitely be extended and refined according to suggestions and experience.

List handling has been enhanced by conforming more closely to standard Mac behavior. In particular, in each panel there is now a moveable focus which allows a list or edit box to be the current target which is controlled using a mouse click or the tab key. For example if you have a large number of search results you can click or tab to the list and then use the navigation keys to examine the list (remaining to implement are sorting and searching of the lists and navigation by typing partial names; also moving hilite using the arrow keys and possibly the shift key). You can also select any number of entries (or all using "Select All") and clear them out using the Edit menu clear command (or the Delete key equivalent). This capability is available for any of the lists you see. In the case of a file transfer or connection list item the corresponding connection will be severed as well as clearing the entry. To simply stop a download and leave the entry in place you can select it and click the “Stop” button. Dragging and dropping an entry to the trash is equivalent to selecting it and choosing “Clear”.

Various minor user interface features have also been implemented. For example, once you have set your download and upload locations you can go directly to them by double clicking the corresponding folder icon. Similarly, once a download is finished you can launch the file by double clicking it in the list. To locate the Mactella application you can command click the title of the window to reveal a popup menu of enclosing folders (thanks to Joakim Braun for this feature). As some may have noticed there is quite a bit of drag and drop capability implemented at least partially. This is again due to cTextDragAndDropAttachment, another effort by Joakim Braun. The lists which include many nice features besides drag and drop are based on CSuperOutlineTable by Joseph Chan.

Another last minute improvement is to allow for drag and drop import of host file lists (note: you can also use copy and paste if that is more convenient). These are the lists of IP addresses that are often gathered to provide a list of active gnutella nodes. Select such a list in an application (like SimpleText, or the NotePad) which supports drag and drop. Drag it over the Hosts list in the lower right corner of the network panel and drop it. You might also notice that while I scan that list for connections (using auto-connect) the host list will scroll to the corresponding position. The program also scans the current list obtained from pong packets so the list will not scroll while those addresses are scanned.

Much remains to implement and improve but with this release we are declaring Mactella to be basically feature complete by naming it beta 0.9.0. It appears a considerable improvement in stability can be achieved by varying the behavior of Mactella per connection (ie dropping packets to connections that are not keeping up). The current strategy of treating connections alike and throttling back Mactella as a whole does not appear to be optimal. We also hope to implement a find by content feature including an example gnutella node at our location hosting a Gutenberg Project mirror. More on this as it develops.

Another area where much effort is currently being expended is building a more efficient hash function for the 128 bit GUID's. When I finish implementing Barrett's reduction algorithm that may lead to many fewer hash collisions and better long term responsiveness. Also, now that most of the needed functionality has been initially implemented some more attention can be paid to improving random number generation and some other technical issues where further attention is certainly warranted.

I also want to offer my thanks to people who have sent in bug reports and suggestions. Now that we have reached beta status and correspondingly a source base which should be changing less radically it will be more feasible to take advantage of the bug reports and continue to make using Mactella more of an adventure in a good sense.

This release of Mactella has more substantial changes than any other so far. It is nearing feature completeness. It is also not entirely stable (...like the others were rock solid). But it is much more capable than the current version so it makes sense to get it out there. Bug fixes will follow and be available at our web site with slight changes in the version number. You can use the “Get Version” button in the preferences dialog or about box to check if there is a version more recent than the one you are using.

The most obvious improvement in this release is we finally have upload capability. I would like to emphasize that this is only for data files. This is how PC's function and gnutella is a network of almost all PC's. In the future provisions might be made for handling Mac files that include resources (e.g. applications) when the transfer is definitely Mac to Mac. For now one can always use a utility like StuffIt to package a Mac file that has a resource fork into a pure data file. Other issues remain for future releases.

The other necessary new feature is the preferences dialog. To allow for useful search query results we need to include connection speed which can only be done effectively by asking the user what the available bandwidth is. The preferences dialog allows you to enable auto-connect by default when the application starts and you can set how many connections Mactella will try to keep up. Along with uploads we also allow for incoming connections on port 6346 (to be added to prefs as an edited quantity in a future release RSN). After the maximum number of incoming connections have been established subsequent connection requests are denied. Port 6345 is used for upload requests and again the preferences is used to determine how many simultaneous uploads are to be allowed. Subsequent requests are rejected until previous uploads have concluded.

The firewall question is being stored but not used yet. In a future (next?) release it will be used to filter search results. If both sides are behind firewalls (ie have private addresses unreachable outside their local networks) file transfers are not feasible so the results will be filtered out. On the other hand, if one is not behind a firewall the search results that are on private networks (e.g. addresses of the form 192.168.x.x) can be accomplished using the push protocol which is exactly how the firewall preference will be used in that case.

The upload feature is only effectively enabled if an upload directory has been specified by dragging and dropping the appropriate folder to the folder icon in the upload panel. When that occurs Mactella will scan the folder looking for how many files there are and how much content that entails and displays the results in the panel. If a user drags and drops an entire drive onto the icon there is a possibility that too many files will be specified and performance will become too sluggish. The user is cautioned of this possibility and allowed to back out. But Mactella will attempt to accommodate your decision if you so desire. Be aware though that the entire upload directory will be scanned each time a search query packet is received (ie very often).

A (much more) stable version of Mactella is being made available today (0.7.0). The foundation for the remaining features needed to make Mactella feature complete has also been laid. This is all due to a completely rebuilt packet switching engine. Other specifics include:

Those disclosure triangles in the search and download lists actually work and show added information.

A nasty intermittent bug in the download code has been rectified.

The acquisition of candidate server addresses has been improved so autoconnect should allow for painless connectivity all the time.

A new panel of statistics has been added for those who need to know all that is happening.

Those statistics showed several problems with packet handling which led to the complete rewrite mentioned in the first paragraph.

Today's release is almost strictly a bug fix version. There were two or three really disagreeable problems with the last version (0.6.0) which have been specifically fixed in this version (0.6.1). Considerably more has been done in the interim but the current development version has some stability problems. Rather than release it and exchange one set of bugs for another the source was rolled back in CVS to the previous release and just the bug fixes were applied.

The most noticeable bug had to do with the search capability. When the connections were stable there should not have been a particular problem. But with autoconnect functioning it was not unusual for connections to get initiated at almost any time. When a new connection was attempted the bug would cause the search attempt to quietly abort. That problem is fixed in this version.

A more serious bug had to do with routing. Under fairly common circumstances the first four bytes of a multi-routed packet would get dropped. This caused Mactella to be a source of bad packets. This bug has also been fixed.

Many were wondering what the annoying beep was about in the previous version. The beep announced that a bad packet had been encountered. The packets are still examined but it doesn't bother to make any noise.

It may be a disappointment to some but the voyeuristic search monitor panel has been removed for a couple of reasons. The most important was probably an issue of performance.

I would like to take this opportunity to thank Shadow_Mage for making a most helpful suggestion which led to the resolution of several bugs. By allowing the various panes to exist simultaneously and making just one visible at a time rather than creating and destroying them on the fly, several complications could be avoided. The search function has also had changes to fix bugs.

One new feature has been included which is found in the About box. A “Check Version” button has been added. When it is clicked our web server is queried to find what the currently available version number is. That is checked against the version you are using and you are advised to visit the web site to download a new version if there is an update. More will be done with this in the next release but we really can't wait any longer to get this version out.

I would again like to thank all the people who have been willing to take the time to send feedback about the product. We can't incorporate all the suggestions but it is very useful to hear about the experiences of as many as we can.

I may as well admit here that I have some reservations about the open source slogan of release early and release often. But we are releasing a new binary today. Our current plan is to release a new binary every week, probably on Tuesdays. If you have Metrowerks Codewarrior PR5 available and a disposition to use it you will find the source code on our cvs server updated daily as it has been for some time.

This release is generally just a “cleaned up” version of our initial release. The architectural changes have been kept to a minimum to decrease the possibility of introducing new bugs. The visual feedback which was so sorely missing in the previous release has been rectified. The larger of the two lists in the “Gnutella Net” panel now just lists current connections. We wait no more than two seconds for a connection to be made so after that time if an address remains that means we are connected.

Probably the biggest difference from the earlier version is the implementation of auto-connect. The list of captured hosts is stored between sessions in the file “Hosts” in your preferences folder. When Mactella starts it automatically tries to establish three connections from the stored list. By clicking the auto-connect checkbox off you can disable this feature. In particular if you want to maintain fewer connections or set them up manually you would need to turn it off. Another feature that has been implemented is the ability to disconnect. Hilight the connection you want off and click the “Disconnect” button.

Another important difference from the earlier version is a result of learning a hard lesson. Some packets are scrambled or badly formed. In particular it isn't too uncommon for a packet to indicate that it has many megabytes of content. In the past Mactella happily tried to make that space available and waited for that much data to arrive. The current behavior is that if any packet requests more than 64K it is deemed to be damaged, it is dropped, and the connection is dropped. If auto-connect is enabled Mactella will try to find another connection to replace it.

It may be a disappointment to some but the voyeuristic search monitor panel has been removed for a couple of reasons. The most important was probably an issue of performance.

I would like to take this opportunity to thank Shadow_Mage for making a most helpful suggestion which led to the resolution of several bugs. By allowing the various panes to exist simultaneously and making just one visible at a time rather than creating and destroying them on the fly, several complications could be avoided. The search function has also had changes to fix bugs.

Some of the most important remaining bugs concern downloading. Although downloading can and does work (assuming the server is available), it is not currently possible to cancel a download. This will be available soon but it requires sufficient modification that I want to do those to the new protocol handler that is being implemented specifically to allow for symmetric handling of server as well as client capabilities.

Just as before our greatest need is for detailed feedback. Any that you can offer will be appreciated. Since this is an open source project we also are anxious for others to read and offer suggestions about the way things are implemented.

Today some much needed attention has been provided to the CVS server and the code stored there. With judicious changes to the “Ignore Files” and “Export Translators” in MacCVSClient the contents on the CVS server should be less problematic. Before, the CVS server and/or client were doing unfriendly things to several of the files as well as pointlessly requiring the user to download several huge files that the IDE was going to rebuild in any case. The “base” version on the CVS server is the contents of the first tarball of source together with one correction (for the quit bug). I've added one mass file update which rolls in all the new list definitions that provide some visual feedback. The resulting version is not yet stable so no corresponding binary has been posted. Please feel free to compile and play with the current code but note the version if you file a bug report.

1) You have to issue a quit for each connection and one for the application. The mechanism for handling this automatically is not currently functioning. Maybe I can fix it tonight before this binary and source code escape.

2) TCP listeners and the associated connection and thread classes. Currently all connections are outgoing.

3) Associated with the TCP listeners for incoming Gnutella connections we need to implement the ability to handle an http request for a file. So if the TCP connection request is followed by an http GET message we send a file. If it is followed by a gnutella handshake we prepare to route packets.

4) Because (2) is not yet implemented we haven't finished implementing the other aspects of file uploading. Pings are parsed and routed but a pong packet is not sent back. This could be easily added to the existing framework but we would be unable to handle a resulting http Get command until (2) is implemented. The same is true of query packets. We parse and route them but don't yet send a query response packet. Again this can be easily added when we are ready to upload.

5) Push packets are detected but not handled except for routing. This might be the next logical feature to add.

6) File type and creator. I suspect there may be some very useful MIME information included with the reply to the http GET together with file name extensions that might allow me to infer correct settings for Mac type and creator. I might also find it necessary to maintain a framework for specifying how the user wants these matters might be determined. The only thing implemented so far is to assume an mp3 filetype for downloaded files.

Major features probably not working right yet
7) I've become suspicious that my mechanism for storing and fetching GUID's may be significantly flawed (ie not working). I can easily identify reply packets to my ping and search query but that only involved a simple block compare. It is the list of GUIDs from pings and search queries that I'm currently investigating. This appears to be responsible for a memory leak (fixed 5/25/00).

8) How many stars are there in the sky? I won't come close to covering all of them but here are some.

9) Visual feedback in the lists on the Gnutella Net pane. If you double click entries the program will try to connect but I haven't implemented visual feedback yet (eg progress of connection attempt, rejections, timeouts, disconnects). The only lame indicator is the spinning arrows. They will spin when a connection is established.

10) VSearch Monitor is just showing the search term, not the speed yet. I'm only showing the most recent 1,000 search terms and lopping off the oldest hundred every time we reach the limit. User specified preferences will be made available to customize behavior. The list behavior when the oldest hundred are dropped is a bit goofy and needs attention.

11) Uploads, there are NO uploads, yet. Currently the app is only a client. (see (1) and (2) above)

12) Although it is perfectly acceptable to download any number of files simultaneously, I am only providing feedback at the top of the list in the Download pane. This needs to be incorporated into the list definition so each entry has its own feedback (and visual progress monitor). I'm using an hierarchical list with the idea that the disclosure triangle will be used to expose the extra information.

13) Search works but has several flaws. When changing panes using the tab buttons the lists need to be reloaded. The captured hosts and search monitor lists seem to work fine but there is still some issue with the same procedure with the search result list. To download just double click the search result you want to download. You are moved to the download pane where you can watch the progress (ie see if you actually get a connection). Although there are cancel and clear buttons they aren't implemented. If I try to implement anything more before releasing this I'm going to be shot.

14) I don't know if this should be major or minor. When I read the packet stream, I grab the header, the added length parameter and the prescribed added amount without looking at it. That packet is sent off to another thread using a queue of handlestreams. When I try to parse the results I sometimes appear to get a packet that does not parse correctly for me which shows up as a -19 error in the debug version of the program (attempt to read past the end of the handlestream). This probably causes the non-debug version to crash. Sorry. There needs to be a robust way to detect and reject packets which don't seem to parse. We may also need to have a procedure for disconnecting and reconnecting when parsing appears to be failing or at least a clear strategy for resynchronizing.